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Abstract 

We consider signal reconstruction from the norms of subspace components. If 
the weighted linear subspaces form a tight p-fusion frame and a cubature for- 
mula of strength 4, then even in case that p of the subspace norms are erased, 
we find a finite list of potential signals, one of which is the correct one. More- 
over, we present a computationally feasible algorithm to determine this list. 
Alternatively and to reduce the number of required measurements to a linear 
scale in the ambient dimension, we use semidefinite programming and random 
subspaces. 



1. Introduction 



Frames have become a powerful tool in signal processin g th at can offer more 
flexibility than orthogonal bases, cf. [IM S3, EJ and [H, EE Hi ^ • Many signal 



processing problems in engineering such as X-ray crystallography and diffrac- 
tion imaging require signal reconstruction from the magnitude of its frame coef- 
ficients, also known under the terms phase retrieval and reconstruction without 
the phase, cf. 0, EH and references therein. By making additional assumptions 
on the underlying frame, exact solutions are presented in 0, see 45, H(| 



for relations to quantum measurements. Recently, reconstruction from frame 
coefficients without phase has been numerically addressed via semidefinit pro- 
gramming, see fll, 13 1 . 



Frame coefficients can be thought of as projections onto 1-dimensional sub- 
spaces. In image reconstruction from averaged diffraction patterns by means of 
incoherent addition of fc wavefields [32j], the original signal must be recovered 
from the norms of its fc-dimensional subspace components. Notably, the latter 
is a common problem in crystal twinning [24|. Here, we pose the following ques- 
tions: Can we reconstruct the original signal from the norms of its fc-dimensional 
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subspace components by means of a closed formula? Also, can we reconstruct 
even if a certain number of norms of subspace components are erased? 

In this note we shall provide an affirmative answer in the following sense: 
Given fc-dimensional linear subspaces {V,}™ =1 in R d , we aim to reconstruct the 
signal x G S"^ 1 from any subset of {||iVj . (^OIIIjLi that has cardinality at least 
n — p. Here, Py denotes the orthogonal projection onto Vj. Under suitable 
conditions on the subspaces, we are able to compute a finite list of candidate 
signals, one of which is the correct one. 

Finding a set of candidate signals is known as list decoding and was intro- 
duced in [31|. We determine this list in a two-step procedure. Without loss 
of generality, let us suppose that the first p norms were erased. If there are 
positive weights such that {(Vj , uij)}^ =1 forms a tight p-fusion frame 

as recently introduced in then we are able to reconstruct the erased norms 
{\\Pv 3 -X x )\\}j=i at least up to permutations. Here, we need to verify that a 
certain system of algebraic equations has only finitely many solutions. Notice 
that our input are not the subspace components but their norms, as opposed to 
signal reconstruction under the erasures discussed in [lj| 14 , 4(J 43 1 . 

In the second step, we assume that {(Vj , yields a cubature formula 

of strength 4, see [H, enabling us to reconstruct the orthogonal projection onto 

from knowledge of {||-FVj ( a; )l|}j = i- This extends the 1-dimensional results in 
[6] to fc-dimensional projections addressed in the present paper. Note that the 
authors in @ require cubature formulas for the projective space whose weights 
are ujj = 1/n, i.e., so-called projective designs. In practice, however, the choice 
of subspaces may underlie some restrictions that prevent them from being a 
design. Nevertheless, given subspaces sufficiently dense in the Grassmann space, 
nonnegative weights can be computed so that we obtain a cubature of a certain 



strength depending on the density to begin with, cf. [33J, [3J, |3jj. Thus, our 
results in Step 2 are a significant improvement for 1-dimcnsional projections 
already. 

The cardinality of a cubature formula of strength 4 scales at least quadrati- 
cally with the ambient dimension d. For k = 1, it is known that scmidefinite pro- 
gramming yields signal recovery with high probability when the 1-dimcnsional 
subspaces are chosen at random [l3| • The cardinality of the subspaces can then 
scale linearly in the ambient dimension up to a logarithmic factor. This fac- 



tor has recently been removed in [12j. Here, we shall extend such results to 



general fc-dimensional subspaces chosen at random. Our proof is guided by the 
approach in fl3l ] but some steps are more involved and need to be taken care of 
with different tools in this more general setting. For instance, the case k = 1 
relies on matrices whose entries are i.i.d. Gaussian modeling the measurements. 
For k > 1, we must deal with measurement matrices involving orthogonal pro- 
jectors that clearly have dependent entries. Hence, the extension from k = 1 
to k > 1 is not obvious and requires special care. The stability under noise, on 
the other hand, follows from the lines in [l3|. We also verify numerically that 
semidefinite programming enables us to recover a signal from the norms of its 
random fc-dimensional subspace components. 
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The outline is as follows: In Section [3J we recall fusion frames and state 
the problem. In Section [3l we introduce tight p-fusion frames and cubature 
formulas. We present our reconstruction algorithm in Section 21 Scmidcfinitc 
programming for signal reconstruction is addressed in Section [SJ and its stability 
under noise is discussed in Section [5] In Section [7J the numerical experiments 
using scmidcfinitc programming arc presented. Conclusions are given in Section 
1 

2. Fusion frames and the problem of reconstruction without phase 

Let Qu.d = Gk,d($) denote the real Grassmann space, i.e., the fc-dimensional 
subspaces of M. d . Each V <G Gk,d can be identified with the orthogonal projection 
onto V, denoted by Py. Let {V,}™ =1 C Qk,d and let {wj}" =1 be a collection of 
positive weights. Then {(Vj , u)j) }"— i is called a fusion frame if there are positive 
constants A and B such that 

n 

A\\x\\ 2 < J2^j\\ P Vj(x)\\ 2 < B\\x\\ 2 , for all x E R d , (1) 

3=1 

cf. [l5|. The condition (UJ) is equivalent to 

n 

A<^2 Uj(P x ,Pvj) < B, for all x e S^ 1 , (2) 
i=i 

where P x is short for P x ^d and (P x ,Pv ) '■— tr&cc(P x Pv ) is the standard inner 
product between self-adjoint operators. If A = B, then {(Vj,ujj)}" =1 is called 
a tight fusion frame, and any signal x G S^ 1 can be reconstructed from its 
subspace components by the simple formula 

1 " 

•'• -yX^'' 1 '-'-- ( 3 ) 

If, however, instead of {Pv}(^)}"=i we only observe the norms {||-Py, (^)||}^ = i 
and, worse, we even lose some of these norms, can we still reconstruct xl In 
the present paper, we find conditions on {(Vj,u;j)}™ =1 together with a computa- 
tionally feasible algorithm such that we can determine a finite list of candidate 
signals, one of which is the correct one. 

3. Tight p-fusion frames and cubature formulas 

3.1. Tight p-fusion frames 

Let {Vj}™ =1 C Gk,d and let {wj}™ =1 be a collection of positive weights and p 
a positive integer. Then {(Vj ■, Wj)}™ =1 is called a p-fusion frame in [5[ if there 
exist positive constants A and B such that 

n 

A\\x\\ 2p <J2^\\PvA x )\\ 2p < B M 2p , forallxeMf (4) 
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If A = B, then {(Vj ■, Wj)}" =1 is called a %/i£ p-fusion frame. As with (JXJ) and 
(|2|), the condition (|4|) is equivalent to 

n 

A<J2 Uj( p x> p v j ) p < S, for all x G S"* -1 . (5) 

3=1 

If {(Vjf, cjj)}" =1 is a tight p-fusion frame, then it is also a tight ^-fusion frame 
for all integers 1 < £ < p, and the tight ^-fusion frame bounds are 

A '-mr,p'- (6) 

where we used (a)i = a(a + 1) ■ ■ ■ (a + £ — 1), cf. We also refer to Q for 
constructions and general existence results. 

3.2. Cubature formulas 

The real orthogonal group 0(M. d ) acts transitively on Gk,d, and the Haar 
measure on 0(M d ) induces a probability measure a k on Gk,d- Let L 2 (Qk.d) denote 
the complex valued functions on Gk.d, whose squared module is integrable with 
respect to a k . The complex irreducible representations of 0(R d ) are associated 
to partitions fx = (li\, . . . , fid), Mi > • • • > AM > 0, denoted by V^, cf. [37[. Let 
be the number of nonzero entries in fj. so that 

L 2 (G k .d)= Hl% where Hft-V*", (7) 

l{fi)<k 

see [13 • The space of polynomial functions on Qj.,d of degree bounded by 2p is 

Pol< 2p (G k . d ) := ff^, (8) 
i(ji)<k, \»\< P 

and we additionally define the subspace 

Pol^2p(e M ):= ffft- (9) 
i(m)<i> l/*l<P 

Let {Vj C 5fe,d and {wj}™ =1 be a collection of positive weights normalized 
such that Y^ij=i w j = 1- Then {(Vj,u>j)} 1 j =1 is called a cubature of strength 2p 
for Q k , d if 



„ n 

/ /(^)dcrfe(V) = ^2<jJjf(Vj) for all / G Pol< 2p (£? M ). 



(10) 



Grassmannian designs, i.e., cubatures with constant weights, have been studied 
in P, 0, S 0| • L° r existence results on cubatures and the relations between p 
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and n, we refer to [22[. It was verified in [B| that {(Vj ■, is a tight p-fusion 

frame if and only if 

~ n 

/ f(V)d* k (y) = Y,'»jf(V j ) for all / G Po\^ 2p (g k4 ). 

Thus, any cubaturc of strength 2p is a tight p-fusion frame. The converse 
implication does not hold in general except for p or k equals 1. 



4. Algorithm for signal reconstruction from magnitudes of incomplete 
subspace components 

Let {(Vj, be a tight p-fusion frame with 5Zj=i u j = 1 ana " ^ 

x € S^ -1 . We aim to reconstruct ±x from only n — p elements of the norms 
{||Py j .(a;)|| 2 }™ =1 . Without loss of generality, we assume that the first p norms 
have been erased, so we want to recover ±x from the knowledge of { || Py i (x) || }™ = „ +1 . 

In a first step, we attempt to compute the missing values tj := \\Pvj (x)\\ 2 , 1 < 
j < p. The second step is dedicated to reconstructing ±x from {||-Pv, C^)!! 2 }^!- 

4-1- Step 1: reconstruction of the erased norms 

The tight p-fusion frame {(Vj, Wj)}™ =1 is also a tight ^-fusion frame for 1 < 
i<p, so that ([6]) yields 

Therefore, (t\, . . . , t p ) is a solution of the algebraic system of equations 



i' 



J=l V 7 ' 3=P+1 

in the unknowns (Ti, . . . , T p ). In the special case of equal weights, ([Sj) gives the 
values of the symmetric powers Yl^—i tj, for I = 0, .. . ,p, which, as polynomial 
expressions, generate the ring of symmetric polynomials up to degree p. Vieta's 
formula yields 

p v 

l[(T - U) = J2(-l) j ejTP-i, where e = 1 and e, = J2 *<i 

i—1 j — l<ii<...<ij-<p 

and Newton's identity leads to 

e^^B-irV^E^ fari = l,...,p. 



3 1=1 3 = 1 
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Therefore, we can compute the coefficients of nf=i(^ 1 — *») as a polynomial in T 
and solve for its roots; we see that (t\, . . . , t p ) is determined up to a permutation 
so that we obtain at most p\ distinct solutions to (J5J . 

If the weights arc not equal, one can still show that fS]) has at most p\ solu- 
tions. The issue is to verify that the associated affine variety is zero-dimensional. 
Results from intersection theory and the refined Bezout theorem, cf. [35, 47 1 and 



[9| , then imply that the variety's cardinality is at most the product of the degrees 
of the p polynomials, i.e., there are at most p\ solutions. 

Proposition 4.1. Let {bi}g =1 be complex numbers and define 

p 

ft(T) = ^2u j Tf-b ii 1 = 1,..., p. 

3=1 

If {u>j}j—i are positive numbers, then the affine variety V := {T g C p : fi(T) = 
0, . . . , f p {T) = 0} is zero-dimensional. 

Proof. We proceed by induction on p. The assertion is certainly true for p = 1 . 
We observe that the Jacobian determinant satisfies 

where we used the well-known formula for the Vandcrmonde determinant. The 
Jacobian determinant is nonzero for T [J i< jAi,j- Therefore, every T g 
V\ j A j.j is a nonsingular point of V, and the dimension of V at T is p — p = 
0, cf. [H Theorem 9.9] and @, Lemma 11.5.1]. It remains to consider the 
intersection of V with the diagonals Ajj := {T e C d : T l = Tj}, i < j. To fix 
ideas, let us consider the case i = 1, j = 2. The intersection V n Ai j2 is given 
by the system of equations 

p 

(wi + uj 2 )T.i + Uj T l = h, 1=1,..., p. 

3=3 

Because u>\ + u>2 > 0, by induction the first p — 1 of these equations have only 
finitely many solutions. Thus, V n A1.2 is finite, too. □ 

We have proved that the system of algebraic equations fS]) has at most p\ 
complex solutions. In order to compute these solutions, standard algorithmic 
methods can be applied 19|, 2(J ■ The construction of a Grobner basis of the ideal 



T generated by the p equations allows to compute the algebraic operations in 
the quotient ring K[Ti, . . . , T p ]/X, which is finite dimensional and of dimension 
at most p\. The computation of the solutions then boils down to linear algebra 
in this space. 



G 



4-2. Step 2: reconstruction from the magnitude of sub space components 

In the second step, we try to compute P x from each of the possible candi- 
dates for {H-FVj (^)|| 2 }" = i- To that end, we establish the following result that 
generalizes the case k = 1 treated in Q. We point out that we allow for cuba- 
tures as opposed to projective designs in Q that require the cubature weights 
to be constant: 

Proposition 4.2. Let {(Vj ;, Wj)}" =1 be a cubature of strength 4 for Gk,d- If 

x G S 4 ' 1 , then 

1 " R 
a * — ' a 

, _ 2k(d-k) j a _ k(kd+k-2) 

wnere a — d ( d+2 )(d-i) ana " ~ d(d+2)(d-i) ■ 

Proof. For any x,y € S"* -1 , the function V n- (P x , Py) (P y , Py) belongs to 
Pol<4(<7fc i( 2). Applying the cubature formula yields 

n 

J2^(Px,Pv j )(Py,Pv j )= (P x ,P V )(Py,Pv)do- k (V). (12) 

j—l JQk.d 

The function 

G:(Rx,Ry)^ f (P x , P v )(P y , P v )da k (V) (13) 

belongs to L 2 (Gi,d x <?i,d) and is zonal. For each variable, it has the form 
Rx h+ (P m ,A(y)), 'where = d (P B , P v )Pyd<7 fc (F), and h+ (P tf ,i4(s)), 
respectively. Since A(y) is self-adjoint and hence a linear combination of pro- 
jections, G(-,y) and G(x, •) belong to Pol<2(<?i,d). The zonal functions on the 
projective space are polynomials in the variable (P X7 P y ) = (x,y) 2 , so that G 
must be of the form a(x,y) 2 + [3. Thus, (fT2|) yields 

n 

£ <*>j (Px , P Vj ) (P y , P Vi ) = a(P x , Py) + (3(U, Py) . (14) 

J=l 

Since p4l holds for every y, we derive 

n 

£ Wj (P a , Py,)Py, = aP a + /3 Id . (15) 
i=i 

Taking traces in (fT5j) leads to fc X)j=i ^jiPx, Py,) = " + and the property 
of tight 1-fusion frames gives X)j=i u j(Px, Py;-) = At = fc/d, so we obtain 

a + d/3 = k 2 /d. (16) 
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Taking x = y in (fl"4"|) implies Y^j=i u 'j (Px , Pvj ) 2 = ce + /3, and the tight 2-fusion 
frame property leads to Y^j=i ^jiPx, Pvj) 2 = A2 = k(k + 2)/(d(d + 2)), so that 
we obtain 

a + (3 = k(k + 2)/(d(d + 2)). (17) 

Solving for a and f3 in (|16j) and (|17[) yields the required identity. □ 

To summarize, we have proved: 

Theorem 4.3. Let {(Vj, Wj)}™ =1 6e a tight p-fusion frame that is also a cubature 
of strength 4 /or C/fc i£ 2. For x E S^ 1 , Algorithm 1 outputs a list L of at most 
2p\ elements of containing x. 

Algorithm 1 List reconstruction 

Input: {tj :HI^(x)|| 2 }" =p+1 . 
Output: L, x € L. 
l: Initialize L = %. 

2: Compute the set 5 of solutions of the algebraic system of equations in the 
unknowns T\, . . . , T p : 

j=i j=p+i 

3: For every (ti, . . . , t p ) € 5, and a, (3 defined in Proposition 14.21 compute 

1 ™ « 
P = -VwAiV - - Id. 

3=1 

4: If P is a projection of rank 1, compute a unit vector £ spanning its image 

and add ±£ to L. 
5: return L 



Note that the actual output list L can be much shorter than 2p\. In the first 
place, we can exclude those solutions of (JSj) that are not real or have negative 
entries. Moreover, in step 4 one can expect that, for most solutions of fS]), 
the symmetric operator P is not a projection of rank one. Also, the solutions 
(t\, ...,t p ) of S that do not satisfy \t\ ' 2 — t l . ' 2 | 2 < \\Pv, - Pv 3 \\ 2 , for every i < j, 
can be removed after step 3 because they violate the consistency conditions 

\\\P Vi {x)\\ \\Pv^)\W 2 < WPvM-PvMf < WPv, -Pvjf, 

where \\Pvi — Pvj II denotes the strong operator norm of Py t — Py j . 

Remark 4.4. 1. For p = 2, the assumptions in Theorem 14.31 reduce to 
{(Vj, Wj)}" =1 being a cubature of strength 4 for Gk,d- Even for k = 1, 
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our result extends [6| since we only need n — 2 elements of the collection 
{\\Pv 3 :(£)|| 2 }j=i as opposed to all n elements in @. This additional flexi- 
bility is not for free: We must assume that x £ S , and, instead of the 
two possibilities ±jc in [f|, we obtain a list L of 4 elements, one of which 
is x. 

2. Since any symmetric matrix X £ M. dxd can be written as a sum of weighted 
orthogonal projectors, (jTTJ) can be extended to 

1 " 8 
X = - 1 S^u; j (X, P Vl )Pv, - - trace(X) Id . (18) 

3=1 

For x £ K d and X = xx*, the tight- 1 fusion frame property yields 
tracc(X) = ||.t|| 2 = | J2'j=i ^ill^^H 2 , so that the entire right-hand side 
of (JT5]) can be computed and hence ±x can be recovered. Moreover, if V 
is a random subspace, uniformly distributed in Qk.d, i-c, according to Cfc, 
then the proof of Proposition 14. 21 yields that 

E(-(X,Py)P y -^trace(X)Id)=X, (19) 
a a 

for all symmetric X £ R dxd . Thus, if {V£}" =1 C Q k ,d are independent 
copies of V, then the law of large numbers implies 

1 ™ 1 " 3 

- V - > (X, Pv^Pvi ~ - trace(X) Id -> X almost surely. (20) 

3=1 3=1 

However, n must be chosen large to obtain an accurate representation of 
X. In Section [3 we shall see that the random choice of subspaces can 
be efficient when the algebraic reconstruction formula is replaced with a 
scmidefinitc program. 



5. Replacing Step 2 with semidefinite programming 



Wc assume in Proposition 14. 21 that the weighted subspaces form a cubaturc 
of strength 4 for Gk,d- However, any real cubature of strength 4 requires at least 
|d(d+l) subspaces, see (22[- Hence, the cardinality scales at least quadratically 
in the dimension d of the ambient space. In this section, we replace the algebraic 
reconstruction formula with a semidefinite program similar to the approach in 



13|, where the case k = 1 was discussed. 

Let Jtf? denote the collection of symmetric matrices in R rfxd . For {Vj}™ =1 C 
Gk,d, we define the operator 

T n :J^^R n , X ^^((X,P V] ))] =1 . (21) 

For x £ S d -\ let / := f (\\P Vj (a:)|| 2 )^ =1 = T n (P x ) £ M" , and we now aim to 
reconstruct x from /. By assuming that the union of the subspaces {Vj}" =1 
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spans M. d , clearly, P x is a solution of 

min (rank(X)), subject to F n {X) = f, X y 0. (22) 

The notation X y stands for X being positive semidcfinitc. Rank minimiza- 
tion is in general NP-hard. Therefore, (f2"2"j) is commonly replaced with 

min (trace(X)), subject to F n (X) = /, X y 0, (23) 
a semidcfinitc program, for which efficient solvers such as interior point methods 



are available on the NEOS Server [21[. For k = 1, there is a constant c > 0, 
such that the random choice of at least cd log (d) subspaces yields that, with 
high probability, the solutions to (f2"2"j) and (|2"3"|) coincide and are unique, cf. [l3j |. 
Recently, the logarithmic factor was removed in [12], so that the cardinality of 
subspaces scales linearly in d. Here, we extend the result to k > 1: 

Theorem 5.1. There are constants c%,C2 > smc/i £/iat, if x E R d , n > cid 

and C Qk,d are chosen independently identically distributed according 

to (Jk, then with probability 1 — e~ C2 ^ the matrix xx* is the unique solution to 

(J23D- 

To verify Theorem 15. 1[ wc shall first derive deterministic conditions serving 
equivalence of both problems (f2Tj) and ([22]) . Later, we shall verify that these 
conditions arc satisfied with high probability when the subspaces are chosen in 
the appropriate random fashion. We follow [l6| and define, for X G Jtf, 

T{X) = {Z - X : Z G JT, Z y tracc(Z) < tracc(X)}. 

A direct computation yields a result stated in : 

Proposition 5.2. T/ie matrix X is the unique solution to (|23|) if and only if 
null(J- n ) n T(l) ={0}. 

While Proposition l5.2l is sufficient and necessary, we also prepare for another 
sufficient condition that appears easier to check for random subspaces. Let e = 
(1,0,..., 0) T G R d and T be the linear subspace {ey* +ye* : y € R d } C Jf. For 
Z G M. dxd , denote Zt its orthogonal projection onto T and Y T ± its orthogonal 
projection onto the orthogonal complement of T. Standard duality arguments 
from linear programming yield the following observations: suppose that there 
exists Y be in the range of J 7 * and consider the conditions 

(D-a) I- Y y and Y T = ee*, 

(D-b) the right lower (d — 1) x (d — 1) block of I t t — Y t t is positive definite. 

If (D-a) is satisfied, then ee* is a solution to (|23p . and if in addition (D-b) is 
satisfied, then this solution is unique. Note that Y is called a dual certificate in 
[I3| and that (D-b) and Y T = ee* already imply I - Y h 0. 

Next, we weaken Condition (D-a) and to compensate, we slightly strengthen 
(D-b) and add a near isometry condition on T n just as in [13(. The proof of the 
following theorem uses Proposition 15.21 and generalizes findings in |13[ . Here, 
|| • ||i denotes the nuclear norm and |j • |joo the operator norm: 
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Theorem 5.3. Let {Vj}^ =1 C G k . d and b = (\\P Vj (e)|| 2 )™ =1 . Suppose that the 

following three points are satisfied, where < r, u k , 7 < 1 are fixed numbers such 

that -y < — 1 and s = r( " fc+1)+(1 ~" fc) 

uiul 7 <^ 2 1+r yjq— ? ana s 1+r 

(1) For all positive semidefinite matrices X € 

-\\MX)\Ui< (1 + r)\\X\\ 1 . (24) 
n 



(2) For allXeT, 

(3) There exists Y in the range of J 7 * such that 



-H^nPOik >«fc(l- ^HJflloo. (25) 

n 



\Y T ~Pe\\HS<l, ll^l|co<i (26) 



Then P e is the unique minimizer to 



The Theorem O was formulated in for k = 1, r = 1/9, u x = 0.94, 
and 7 = 1/3 only. Since < s < 1, the choice 7 = j^j^i f° r instance, 
is admissible. We can straightforwardly follow the lines of the proof in [l3| 
while keeping track of the constants, so we omit the explicit proof here. It is 
noteworthy though that it relies on Proposition l5.2l and that the special form of 
F n is not used, so that the statement holds for any linear map from to M n 
satisfying the conditions of Theorem 15.31 

Condition ((26]) is the modification of the above Conditions (D-a) and (D-b), 
and (|24|) with (j25|) are used to make this modification work. Note that (|24| is 
natural in the setting of random subspaccs. Assuming that {Vj}™ =1 is a tight 
1-fusion frame, we obtain, for positive semidefinite X G Jrff, 

, n d , n d 

ll-^(X)||, 1 = |^(X,Pv i )=^A i -^(P ;EiJ Pv,)=^A i = ||X|| lj 
3=1 <=i 3=1 i=i 

where {Aj}f_j are the eigenvalues of X and {xi}^ =1 the associated unit norm 
eigenvectors. We shall verify later that the random choice of subspaces approxi- 
mates a tight 1-fusion frame, so that (j2"4"]l is satisfied with high probability. The 
corresponding inequality from below then holds too but (|25[) asks for the oper- 
ator norm in place of the nuclear norm and must hold on T. To compensate, 
we allow for an additional constant u k in (|25l) . 

In the subsequent sections, we shall verify that the condition of Theorem l5.3l 
are satisfied with high probability when the subspaces {Vj}™ =1 are selected at 
random. 
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5.1. Nuclear norm estimates on || J r „(X)||£ 1 for X y 

We shall first verify that T n is close to an isometry with high probability: 

Theorem 5.4. For < r < 1 fixed, there are constants c(r),C(r) > 0, such 
that, for all positive semidefinite matrices X and n > c(r)d, 

< ^nPOIk < (1 +011X11!. (27) 
holds with probability at least 1 — eT c ^ n . 



To prove Theorem 15.41 we need to extend a result in 48| on the smallest 
and largest singular values s m i n (P) and s max (P), respectively, in which P has 
independent sub-exponential rows. Here, we consider independent blocks but 
there may be dependent rows within each block: 



Proposition 5.5. Let P := W f (Pvi , ■ • ■ , Pv n ) be a random matrix, in which 
{Vj}™ =1 are identically and independently distributed according to on Sk,d- 
Then, for every t > 0, we have with probability at least 1 — 2exp(— ct 2 ) 

- CVd - t < s min (P) < s max (P) <Vn + CVd + t, 
where c, C > are absolute constant. 

The proof of Proposition 15.51 is given in |Appendix A| 



Proof of Theorem \5.4\ Since any positive semidefinite matrix X can be written 
by means of its projectors on eigenspaces, it is sufficient to verify 

1 - r < -\\TJxxn\U, < 1 + r, e £ d_1 , 
n 

in place of (|2T|) . We observe that ||Pa;|| 2 = || T n (xx*)\\e 1 , so that 

S 2 min (P) < WFnixX*)]]^ < 4^(P) 



holds. First, we take care of the upper bound. According to Proposition 15 .51 
we have 

"ll-Fn^k < -sl^(P) < (1 + ^iCVd + t)) 2 , 

n n \Jn 

with probability at least 1 — 2e _ct . Choose e > such that r/4 = e 2 + e and 
observe that e > |, so that n > c\r~ 2 d with c\ = 25C 2 implies n > e~ 2 C 2 d. 
For t = \fne, we obtain that 

-^s max < (1 + 2s) 

holds with probability at least 1 — 2e~ cne . Hence, we have < (1 + r) with 

the same probability. Since e 2 > r 2 /25, we can adjust C\ such that n > ^ hi(2) 
so that C2 > exists and the required upper estimate holds with probability 
1 — e~ C2r ™, The lower estimate can be derived in an analogous way. □ 



12 



5.2. Operator norm estimates on ||J r n (X)||£ 1 for symmetric rank-2 matrices 

We shall verify a lower estimate on ||J r „(X)|jf 1 involving the operator norm 
when X is a symmetric rank-2 matrix: 

Theorem 5.6. There is a constant u k > 0, only depending on k, such that the 
following holds: for < r < 1 fixed, there exist constants c(r),C(r) > 0, such 
that, for any symmetric rank-2 matrix X , n > c(r)d, 

-\\T n (X)\\ ei >Uk(l-r)\\X\\ O0 
n 

with probability at least 1 — e~ c ^ r ' n . 

We need two preparatory results: 
Proposition 5.7. There is a constant u% > such that, for all —1 < t < 1, 

^E\\\P v (e 1 )\\ 2 -t\\P v (e 2 )\\ 2 \>u k . 
Lemma 5.8. The random variable 

£=7ll|/V(ei)!| 2 -t!|Py( e2 )|| 2 | satisfies sup lME|£| p ) 1/p < 2. 

K P >1 

Both, Proposition 15.71 and Lemma 15.81 are proven in |Appcndix B| 

Proof of Theorem \5.6\ It is sufficient to consider H^Hoo = 1, so that X = P u — 
tP w , where u, w G S*- 1 and u _L w and t G [-1, 1]. Since is invariant under 
0(R d ), we can assume that u,w are the first two unit vectors e\ and e 2 . We 
observe 

1 1 " a i " 

-||^(J0lli = -E fcHI^( e i)H 2 -t||Pv;,( e2 )|| 2 | =:-£&, 
j'=i i=i 

where are independent copies of 

C = ||||Py(ei)|| 2 -t||Pv(e 2 )|| 2 |. 

The Bernstein inequality for sub-exponential random variables as stated in [48| 
can be used with Lemma I5~51 to obtain 

P(|-||.F n (jr)||i -E£| >e)< 2cxp(- C nmin(^)), 
n 4 2 

where c > is an absolute constant. For e < 2, we derive with Proposition 15.71 

-ll-FnPOHl^Ufc-e, 

n 

with probability at least 1 — 2 exp(— Cine 2 ), where C\ = c/4. The choice e = u^r 
establishes the required estimate at least for fixed X <G T with probability at 
least 1 — 2 exp(— C 2 nr 2 ), where C 2 = G\u\. The remaining part of the proof is 
the same covering argument as in [131 ] . so we omit this. 

□ 
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5.3. Dual certificate: Yr 

It is not hard to see that T* is given by 



d 



i=i 

so that T^TniX) = p- Xw=i(-^"> Py.)Pv } . If V is a random subspace in Qk,, 
distributed according to <jk, then p9|) yields, for all X e Jf, 



if 

k 



2 E(X,P V )P V = aX + 6trace(X)7, (28) 



where a = k ^2)(d~i) an< ^ ^ = k(d+2){d-l) • ^ s pointed ou t m (|2"0|) . the law of 
large numbers implies that T^fF n tends to S, where 

S -> , X^aX + fo tracc(X)/. 

We can compute its inverse as 

S" 1 : JT, X H- -X - 4- tracc(X)/, 

a da 

where the special form of a and b is used. Thus, J 7 * J^iS" 1 tends to the identity, 
which motivates the choice of a candidate for Y as 

1 1 d 2 " 

y = -JSJWTV) = £<S-i(ee*),Pv 3 .>iV, 

In view of tail bound estimates, it will be advantageous though to use an addi- 
tional cut-off: 

Theorem 5.9. Given 7 > ; let {V^}™ =1 be identical and independently dis- 
tributed according to o k on Qk,d an d define Yj = {-i~\\Pvj ( e )|| 2 — a)f^V,- Let 
Y = ^E^i^j 1 ^, w/iere 

^■ = {\/fll^(e)||<2/3 7 } (29) 

and /3 7 > a sufficiently large constant. Then Y is clearly in the range of J 7 * , 
and there are constants c 7 , C 7 > such that, for n > c 1 d, 

\\Y T ~ P e \\HS < 7 (30) 
holds with probability at least 1 — e ^ c i n / d . 
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Remark 5.10. The structure of Theorem l5.9l and its proof resemble the lines in 
13[ but we removed the log(d) from the constraint in Ej. It enables us to remove 



the logarithmic factor from the lower bound on the number of measurements. 
Alternatively, Candes and Li removed this factor for k = 1 using a slightly 
different construction in [12 1. 

The proof of Theorem 15.91 is based on two propositions: 

Proposition 5.11. Under the notation of Theorem \5.9l let V be a random 
subspace distributed according to Cfc on Gk,d- Then we have, for all j3 1 > 1/2, 



f ||Py(e)|| > 2p y ) < 2e k ^e- k ^. (31) 



Proposition 15 . 1 ll immediately follows from Vershynin's lecture notes on non- 
asymptotic random matrix theory, (Lemma 9 in Lecture 4 on dimension reduc- 
tion). 

Proposition 5.12. Under the notation of Theorem \5.9\ and n > c 7 d, let 

v 3 = 4 Pv * e - e > & = ^" Py »n 2 - - a (32) 

Then we obtain, with probability 1 — e -C 1 n/d ^ 



1 



1 ™ 

We shall verify Proposition 15.121 in |Appendix C| 
Proof of Theorem \5.9l Since \\Yt — PcWhs < V^Ire — e||, we shall use 



YTe - e= n^ y]lE >~n^ elE i ' (33) 

where yj is defined by (|3^|) and Ej denotes the complement of the set Ej in the 
appropriate probability space. We shall estimate both terms at the right-hand 
side of (|33|) separately. Let us denote 

7r(f3 J ):=n^\\P v (e)\\>2P 1 ). 

We apply Hoeffding's inequality to obtain 
1 " 

p {\-^2 1 E<; - ttW 7 )\ >t) < 2cxp(-2nt 2 ), V t > 0. (34) 

U 3=1 
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According to Proposition 15. Ill we can choose /3 7 sufficiently large such that 
7r (/?7) < 4^/2- F° r t = -^pfli the estimate in (|34|) implies that 

2 

holds with probability 1 — 2e~Te n . Combining the latter with Proposition [5712] 
yields the estimate (|3T))) with the required probability. □ 

5.4- Dual certificate: Y t t 

Recall that Y = ± ££ =1 ^ with Y 3 = I(f ||P,,( e )|| 2 - b)±P Vj and E 3 = 

{yf ||-FVj( e )|| < 2/3 7 } were defined in Theorem l5.9l This section is dedicated to 
verifying the following theorem: 

Theorem 5.13. There are constants c, C > smc/i £/ia£, /or n > cd, 



\Y T ±\\oo<l (35) 



u;it/i probability at least 1 — e 



-Cn 



The random variable ||Py(x)|| 2 does not depend on the choice of x € S 1 
and is beta distributed with parameters (|, ^p). Its moments are given by 

FIIP ( Mi2p ( fc / 2 )p fc(fc + 2)---(fc + 2p-2) 

E||Pv(a;)l1 = d(d + 2)--(d + 2p-2) ' (36) 

which coincide with the tight p-fusion frame bounds ([5]) when the weights are 
constant. Equation (|56"|) is the first step to verify the following lemma whose 
proof is given in |Appcndix D| 

Lemma 5.14. If V is a random subspace in Qk.d, k>2, distributed according 
to <7fc, then we have, for any u € 5 fd_1 and any integer p > 1, 

d" 



:P vm\Mu<w p <2 p P \. 

We need an additional lemma involving nets. A (sperical) e-net Af £ is a finite 
subset of S^ -1 such that to any clement x £ S d ~ l , there is an element in J\f e at 
distance less than or equals e. 



Lemma 5.15 (4a, Lemma 5.4]). Let A <G R dxd be symmetric, and let M e be 
an e-net of S d ~ 1 for some e € [0, g). Then 

\\A\\ OD = sup \{Ax,x)\ < (l-2ey 1 sup \(Ax,x}\. 
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Proof of Theorem[5jM We see that Y T ± = ^YTj=i X j l E v where Ej is given 
by (|29p and Xj are independent copies of 

X~l(l\\Pv(e)\\ 2 -b)l(P v ) T ,. 

In order to verify ([551) with high-probability, we shall show that || i X)j=i -^j II — 
| holds with the required probability, where Xj = Xj 1 e , ■ We first write 

n n n 

\\J2 x j\\= sup \^2(u,Xju)\= sup IYIvAejI (37) 
where 77.,' are independent copies of 

V=(^\\Pv(e)\\ 2 --)j(u,(Pv)T^u)). 
ka a k 

Note that we shall take care of the dependency of 77 on u with a covering argu- 
ment at the end of the proof. In order to apply a Bernstein inequality, we shall 
estimate, for an integer p > 0, 

E\r,lE\ P = E [^(^!l^( e )H 2 - b) P ^;(u, (iV) T .L («)>*1 B 

The Cauchy Schwartz inequality yields 
EM B r<yE[^(^||F v .( e )|p-6) 2p l £ ]yE[^( u ,(Pv-)T-H> 2p l £ ]. (38) 



Next, we estimate the two terms on the right-hand side separately. First, we 



derive, for sufficiently large /3 7 , that || ||Py(e)|| 2 — b\\ E < 4/3 2 , so that 



E|^||Pv(e)P-&ri JJ < (0$y. (39) 

To estimate the second term on the right-hand side of (|5"8"|) . we observe 
that (u,(P v ) T ±(u)) < \\P v (u)\\ 2 . We now combine Lemma [5TT41 with (|3"9"]l to 
estimate (J3HJ) by 

E^i^r <(^f P !=(^)V(-^r 2 . 

a ' a ' a ' 

The Bernstein inequality as stated in [l^, Theorem 4.4], for instance, then yields, 
for all t > 0, 

P(|$>1 B , -nE % l % | > „i) < 2exp(- 4(g ^^ +lg ^ ). 

Therefore, there is a constant Cp > such that, for all i > 1/n, 
1 " 

Pd-^^lB, - Er7jlj3.| > t) < 2cxp(-C (3 ni). (40) 
71 i=i 
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To estimate jE^Ts, |, we first observe that j^Pv = I and evaluating ([25)1 at 
P e yields 

E7 ? = — E(Pv,Pe)((Pv) T TU,u) - —E((P V ) T TU,U) 

ak z ak 

= —((aP e + bI) T T u,u) (Irpru, u) = 0. 

a a 

Since ([36]) implies E77? < c, for some constant c > 0, Cauchy-Schwartz and (|31[) 
imply 

|E % l Bj | = |E % l B c| < ^¥{Efj^^< v /ot(/3 7 ) < Vc2e fc /2 e -fe^. 

Therefore, (gDJ yields 
^ n 

P(-|^r/ i l i5j | > i + V / c2e fe / 2 e" fe ^) < 2exp(-C /3 ni), 

for all t > 1/n. If c 7 > is a sufficiently large constant and /3 7 is sufficiently 
large too, then n > c 7 and < = ^- yield 

n 

P (-|E^'I ^ V4) < 2exp(-C», (41) 
n 3=1 

where C 7 > is some constant only depending on 7. 

Next, we apply a covering argument to obtain a bound on || Y]j—i X-j\\ in 
(j3"T)l : let A/1/4 be a 1/4- net on S^ 1 , so that Lemma T5 . 1 5 1 implies 

n 1 n 1 n 

H~y]^jll = SU P |(«, < 2 sup - VXj"u)|. 

71 3 = 1 u£S d - 1 ,u±e 71 j=l ueNt/i 71 j=l 



The net A/1/4 can be chosen such that | A/1/4 1 < 9 d j cr - 113|j so that (|4*T|) implies 

1 71 1 n 

P(||- VXjII > 1/2) < P( sup I (it, - y2Xju}\ > 1/4) < 9 d 2cxp(-C* 7 n). 

n j=i ueAT 1/4 n i= i 

If c = c 7 is sufficiently large and n > c<i, then we can transfer the factor 9 d 2 
into a constant C > 0, so that 



1 " 

P(||-E^||>l/2)<cxp(-Cn), 
which concludes the proof of Theorem 15.131 □ 
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5.5. Proof of Theorem \5.1\ 

We can now assemble all of our findings to verify that the conditions in 
Theorem 15.31 hold with the required probability: 

Proof of Theorem \5.1\ We fix r e (0, 1) and choose 7 = -^^j^i where Uk G 
(0, 1) as in Lcmma [5.7l Let c; and Cj, i = 1, . . . , 4, be suitable positive constants. 
Theorem 1 5 . 41 yields that Condition ([24]) holds with probability of failure at most 
e~ Cin , for all n > c\d. Theorem 15.61 implies that Condition (|25p holds with 
probability of failure at most e~ C2 ™, for all n > c^d. According to Theorem l5.91 
the first condition in ([2"B]) holds with probability of failure at most e _C3 "/ d , for 
all n > 03d. Theorem 15.131 yields that the second condition in (|2"6"|) is satisfied 
with probability of failure at most e 4 ™, for all n > c^d. 

Finally, there are constants c, C > such that, for all n > cd, we can estimate 
Y^t=i e~ Cin l d < e ~~ Cn / d , so that all conditions in Theorem [531 are satisfied with 
probability at least 1 — e ~ Cn / d . This implies Theorem 15. II □ 

Remark 5.16. Recently, a more general framework for minimization problems 
of the form (f22|) and ([23[) have been considered in [l(|. First, replace T in ([2i~j) 
with T(X) = ((X, Zj))* =1 , where {Zj}* =1 C R dxd is a collection of independent 
random matrices with independent identically distributed Gaussian entries. If 
x G S^ 1 and b = J-(P X ), then results in [l6| imply that, with high probability, 
the solution to ([2U)l is unique and also solves ([2"2")l . These results do not apply 
in our setting though, because we must deal with orthogonal projectors. 



6. Stability under distortions 

We shall verify that our approach is robust against contamination of the 
measurements. For x £ R d and following 13(, we assume we measure 

fi = \\PvA*)\\ 2 + Vj, 

where rj = {r]jYj = i 1S a distortion bounded by \\r]\\g 2 < e, for some e > 0. Wc 
replace (f2U)l with 

min (trace(X)) subject to WFJX) - f\\ < e, X h 0. (42) 

For X being a minimizer of f|42[) . we extract ±i from xx* := X\P Vl , where v\ 
is the unit norm eigenvector of X associated to the largest eigenvalue Ai. 

Theorem 6.1. There are constants Ci, 02,03 > such that, if x £ M. d , n > c\d 
and {V}}"—! are chosen independently identically distributed according to o~k on 
Gk.d, then with probability 1 — e~ C2 5 the solution to (|42[) satisfies 



\\x — cpx\\ < C3 min(||x||, e/||a;||), for some ip G {±1}- 



The proof is derived by following the lines in [l_3j and keeping track of the 
constants. For the sake of completeness, we give a brief outline: 
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Proof. Without loss of generality, we may assume x = e and define R := ee* — X. 
Rescaling and a rotation then yield the general result. Given < r, Uk < 1, we 
choose 7 < i=£ , so that the conditions in Theorem 15.31 arc satisfied with 

' 2V2 1+r' 1 1 

probability f — e~ C2 ^. By following the lines in [l3j |. we derive 

HjRtj.Hi <4e(l + r)+2V2 7 ||i?T||co, (43) 
u fc (l - rJU-Rrrlloo < 2n^ 1/2 e + (1 + r)||i? Ti ||i. 

Merging the two inequalities yield 



4nt(1 - f)(1+r ^ 7) 

u fc (l-r)-2^(l + r)7 

and our condition on 7 implies that the denominator is positive. Thus, we can 
define a constant c\ > such that < cie. Using p3| yields ||-Rt||oo < 

C2£j_for some C2 > 0, so that ||ee* — Xjloo = |P||oo < (2i + c 2 )e. It was verified 
in [13j that the latter is sufficient to deduce the existence of a constant C3 > 
satisfying ||e — <C3min(l,e). □ 



-1/2 



7. Numerical experiments 

We shall present some numerical experiments illustrating Theorem 15. 11 Let 
x E S^ 1 and observe that V £ Qk,d is uniformly distributed if and onl y if 
P v = Z{Z*Z)~ 1 Z* for some Z <G R dxk with independent 7V(0, 1) entries, cf. |l3, 
Theorem 2.2.2]. Thus, we can easily generate pseudo-random orthogonal pro- 
jectors {Py,}™ =1 . Since ||Py.-L (x)\\ 2 + 1 1 -FV^- (a?) 1 1 2 = ||x|| 2 , we shall restrict us to 



k < d/2. We follow the numerical experiments in 13 |, where the measurement 
vector / is given by 

fi = \\PvMT+ r >h (44) 

and r\ = {?7j}™ =1 are independent Gaussian random variables with zero mean 
and variance £„. It then is standard to replace (|23[) with the relaxation 



min(||J"pf)-/|| 2 + Atrace(X)), subject to X>Q. (45) 

As in [l]|, we use the software package Templates for First-Order Conic Solvers 
(TFOCS) [3 to solve (05]). If X is the solution to fl45j, thcn w e define ±x e S^ 1 
as the normalized eigenvector corresponding to the largest eigenvalue of X, 
because we already know that x has unit norm. 



First, we follow a numerical test from for the noise free setting. As in [11 1 
for k = 1, the computed approximation is visually indistinguishable from the 
test signal when k = 10 and k = 20, cf. Fig. [TJ We compare the approximation 
error min(||a; ± x||) for several choices of k in a separate experiment in Fig. [2j 
There, we observe that the error decreases with growing k when A is kept fixed. 
An explanation may be that, as k grows, the terms {||Pvi (^)|| 2 }^=i increase 
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original 
appr oximation 




(a) k = 10 



(b) k = 20 



Figure 1: We choose the original signal x uniformly distributed on the sphere S d—1 and 
the measurements b are noise- free. As in [ill , where k = 1 was used, the approximation is 
computed for d = 128, n = 6a!, and A = 0.05. Here, also for k > 1, we see that original and 
computed signal are visually indistinguishable. 




(a) We chose d = 20 and A = 10~ 6 . The error(b) For d = 100, n = Wd, and A = 10~ 2 , the 
decreases with growing subspace dimension k error decreases with growing k. 
and cardinality n. For k = 1, the error is above 
the range of this plot. 

Figure 2: The signal is x = (1,0, ...,0) T 6 S' 1 - 1 . We computed the error mini \\x ± x ) 
averaged over 100 iterations. Since there is no noise, A should be small. 



so that the ratio between the two competing terms in (|45p changes in favor to 
larger k when A is held fixed. 

Second, we aim to reconstruct the signal i in a noisy setting for several 
choices of S^. The relaxation parameter A must still be adopted to the strength 
of the noise. If is indeed known, then A can be chosen according to the 
discrepancy principle, but, in practice Ej, can only be estimated and cross- 
validation is more commonly used, see [39( for advantages and problems with 
various parameter estimation techniques. Here, we shall adopt the L-curve 
criterion HH, see also [29], 44 1, so that we choose A according to the maximal 



curvature of the two-dimensional curve (||.F(Aa) — .f|| 2 ,trace(X,\)), where X\ 
is the A-dependent solution to (|45|). 

In Fig. |3l we adjust the noise level so that SNR = ^f- is fixed when k 
changes. Here, E/ is the standard deviation of the collection {\\Pv 3 ^(a;)|| 2 }j = i- 



21 




'l 16 32 48 64 80 96 112 128 °' 15 1 16 32 48 64 80 96 112 128 



(c) SNR = 4 (d) SNR = 8 

Figure 3: For the given data vector b, we fix the SNR. We chose n = lOd, and k = 1, 8, 32. 
The original signal x is again a normalized sine wave with d = 128 samples. 

The three choices of k then seem to perform equally well. Thus, we do not seem 
to lose anything by using orthogonal projectors whose rank is not one. 

It appears reasonable to believe that properties of the signal x that are 
known a-priori could be used to add further penalty terms in (|45[) improving 
the reconstruction. 

8. Conclusions 

We derived algebraic conditions on subspaces {Vj}" =1 in Proposition 14.21 to 
reconstruct any signal up to its sign from the norms of its subspace components 
by means of a closed formula. Those conditions require the existence of nonneg- 
ative weights {wj}" =1 such that {{Vj, Wj)}™ = i is a cubature for Qk t d of strength 
4. The latter extends the results in [6[ in two ways: We deal not only with 
one- but with fc-dimcnsional subspaces, and we do not require the weights to be 
constant. Next, we tackled the erasure of subspace components using similar 
algebraic conditions. If {(Vj, Wj)}" =1 is in addition a tight p-fusion frame and 
x G S d ~ x , then {\\Pvj 0*011 2 }j=i can essentially be determined from any subset 
of cardinality n — p, see Proposition 14.11 Indeed, the erased p norms can be al- 
gorithmically recovered up to permutation. Note that any cubature of strength 
4 is also a tight 2-fusion frame so that we only need n — 2 norms of subspace 
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components to recover x. To our knowledge, the presented algorithm is even 
novel for 1-dimensional subspaces. 

The cardinality of cubatures of strength 4 scales at least quadratically in 
the ambient dimension d. For 1-dimcnsional random subspaces, scmidcfinitc 



programming was used in [13| to reduce the number of measurements to a 



linear scale up to a logarithmic factor, and this factor was later removed in 



12j . Recently, an optimizationless approach for 1-dimensional subspaces was 
proposed in [23[ , but the lower bound on the number of measurements involves 
the additional logarithmic factor as in [Hi ]. 

Here, we were dealing with general fc-dimensional random subspaces in Sec- 
tion [7] and verified that the signal can be recovered in a stable fashion by using 
scmidcfinite programming. It is noteworthy that we avoid the logarithmic fac- 
tor and reduce the number of required measurements to a linear scale in the 
ambient dimension. 
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Appendix A. Proofs for Section 15.11 

Our proof of Proposition 15.51 requires a lemma for preparation: 



Lemma Appendix A.l ([43, Lemma 5.36]). If B e W ixd satisfies \\B*B - 
I Woo < max((5, 5 2 ), for some S > 0, then 

1-S< s min (B) < s raax (B) < 1 + S. (A.l) 

Conversely, if B satisfies (|A.1[) . then \\B*B — I\\oo < 3 max(<5, S 2 ). 



Proof of Proposition \5.5\ To verify Proposition 15 .51 we want to apply Lemma 
|Appcndix A.l| with B = 77= -P- First, we take care of the upper estimate on 
Smax(-P)- We must check that 



1 rl n [7ft 

P Vi -III <max(<5,(5 2 ) =: e, where S = C\- + -=. 
n k z — ' V n Jn 

.7=1 



(A.2) 
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Let jVi/4 be a j-net, so that we can estimate with Lemma 15.151 

l|Vp-I|| = ||i!X> 3 --J|| 
n n k z — ' 

3=1 

Id" 

<2 max | ((-T^^y i-/)a?,x) | 

' 3=1 

= 2 max |-||Pa;|| 2 - II. 

ieJVi/4 n 

Thus, we must verify with the required probability that 

max \h\ Px f-l\< s (A.3) 
ajeA/i/4 n I 



To derive this estimate, we define random variables Zj = J j||^V^(x)|| so that 
Ej=i z ) = ll-f^ll 2 - Since E(Z?) = 1, we can estimate 

U^IU := B upp- 1 (E^) 1 /i' 
P >i J 

= ^supp- 1 (E||P Vi ( a; )||^) 1 /f 
fc p >i 

= TSUpp ( ) IP < 1, 

fc p >i (a/2)p 

where the last inequality can be derived by induction over p. According to [48l 
Remark 5.18], \\Zj— l||i/>i ^ 2, and we obtain from the Bernstein type inequality 



48|, Corollary 5.17] 

1 " r\ 1 " 

P ^ E % l|P ^ (2;)l12 _ 11 - £/2) = P(l n E Z I - !| ^ £ / 2 ) 

3=1 71 3=1 

£ 2 £ 

< 2exp(-cnmin(— , -)) 

= 2exp(-n— S 2 ) 
v 16 ; 

<2exp(--^(C 2 d + t 2 )), 
16 

where the last line follows from (|A.2[) . Since the net can be chosen such that 
|A/"i /4 | < 9 rf , cf. 53, we can verify (|A~3]l by 

1 " rl 

P( max |-£ ||P v (o:)|| 2 -l|>e/2)<9 d 2e X p(--(C 2 d + t 2 )) 
ieAfi/ 4 n ' k 16 

3=1 

< 2exp(--^ 2 ), 
16 
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where we assume C > 4i/ln(9)/c. The latter does not cause any trouble because 
c is a constant independent of e. This finally yields 

POw(-P) > VTi + CVd + t) <2cxp(--^ 2 ). 

16 

The estimates on s m i„(-P) are derived analoguously. □ 



Appendix B. Proofs for Section 

We shall prove Proposition 15 . 71 and Lemma IB~8 



Proof of Proposition [577| Since the integral is always nonzero, we only need to 
take care of the limit d — > oo. We first see that 

|E|||P v ( ei )|| 2 -i||Pv-( e2 )|| 2 | = 7 / |||Py( e i)|| 2 -t||Py( e2 )|| 2 |^ fe (y) 



k "' v " v 1 k 



Gk^d 

k k 



d f 

i / \J^ m li- t J2 m h\ d ^ M ^ 



2.<i i=l i = l 

where V 2 .d = {M = (m itj ) G K dx2 : M*M = 1} denotes the Stiefcl-manifold 
endowed with the standard probability measure vi. If M is a random matrix, 
distributed according to v 2 , then, according to [gjj Proposition 7.5], the upper 
k x 2 block of M multiplied by d converges in distribution (for d — > oo) towards 
a random k x 2 matrix whose entries are standard normal i.i.d.. Let us denote 
the underlying probability measure on R fex2 by A/"(0, Ik® It)- The convergence 
in distribution implies that, for d — > oo, 

dE|||P y ( ei )|| 2 -t||P y (e 2 )|| 2 | -> / |||7V( ei )|| 2 -i||7V( e2 )|| 2 |dAA(0,4®/ 2 )(iV). 

jR fc x 2 

Since the right-hand side is bigger than 0, for all — 1 < t < 1, compactness and 
continuity arguments suffice to conclude the statement of the lemma. □ 

Proof of Lemma \5.8[ By using the expressions of E||Py(cc)|j 2p in (|36p . we can 
estimate 



(Em i/p < -(e(hpv(u)ii 2 + \t\\\p v (w)rr) 

= f ( E (^)E(iip y ( U )n 2 " i |tr™iip y Hii 2(p - m) ) v 

m— ^ ' 

( ^ ( n vmPv(u)\\^E(\\p v (wW(p-™ 



k m=0 
/ P 

m=0 



d 2 k{k + 2) d 2 (fc + 4?n-4)(fc + 4m- 2) 
my V fc 2 d(d + 2) ' ' ' fc 2 (d + 4m-4)(d + 4TO-2) 



'd 2 A:(fc + 2) d 2 (fc + 4(p - m) - 4)(fc + 4(p - to) - 2)\ 1 /p 
fc 2 d(d + 2) " ' fc 2 (d + 4(p - m) - 4)(d + 4(p - to) - 2) / ' 
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Now, we check that, for k > 2, I = 0, 1, 2, . . ., 

d 2 (fc + 4l) (fc + 4£ + 2) 41 4^ + 2 

P(d + 4^) (d + 4^ + 2) - (1 + T )(1 + ^ } - 2 ' 

Thus, we obtain 

i/p 



m=0 \ TO / 



/ \ i / ?> 

<2(p!(p+l)J <2p 



where the last inequality holds provided that p > 3. The latter follows from 
p\(p + 1) < p p , for p > 3, which can be shown by induction. Similar arguments 
as above yield 1/2(E|^| 2 ) 1 / 2 < V3 < 2 and E|£| 2 ) < 2 so that we obtain the 
required statement. □ 

Appendix C. Proofs for Section 15.31 

Before we can verify Proposition l5 . 1 2l using a Bernstein inequality, we require 
some bounds on the random vectors yj given in (|32l) and yj :— yjlEj- 

Lemma Appendix C.l. There is a constant c > such that the following 
points hold: 

(a) E(|| % || 2 ) <cd, 

(b) ||ft|| a < cJl%, 



(c) ||Ey|| 2 < cV2e k / 2 e~ k ^. 

Proof, (a) By applying (j3"6")) for p = 1 and p = 2, a short calculations yields 
E£j||-£V(e)|j 2 = an d we can compute 

E(\\ yj \\ 2 ) = E(U^P v e-e\\ 2 ) 

= mf^\\Pv(e)\\ 2 -^\\Pv4 2 + l) 



m\\Pv(e)\\ z )-2- 



k 2 v*jn "v~/m j d _ k 
^ll^(e)f-^) 2 ||P.(e)f)-2^ + l 

-^E((^||P v (e)|| 4 - 2^||P y (e)|| 2 + 6 2 )j|Pv(e)j| 2 ) + 1 

d 2 rd 2 k{k + 2){k + A) bdk(k + 2) b 2 k\ d 
~a T k 2 \k 2 d(d + 2){d + 4) ~ 2 kd(d + 2) + ~d) ~ 2 d-k 



26 



We can estimate a and b by a constant independently of d, which implies (a). 

(b) If £j is given by (|32|). then a direct calculation yields that there is a 
constant c > such that < c/3 2 , holds on the event Ej. Since ||j/j|| 2 — 

e 3 U P vM\ 2 - %A \\PvM\ 2 + 1. we can use Ill^(e)|| 2 < 4/?2 on 25, to 
obtain the rough estimate 

m\ 2 < c^i^, 

where c\ > is some constant. 

(c) The term EPy. is a multiple of the identity, so that the entries of Py (e) 
have zero mean except for the first one. Moreover, (|28|) yields 

^E\\P Vj (e)\\ 2 P Vj (e) = (a + b)e 

and a + b = feS , which implies that the entries of yj = \\ Pv, (e) || 2 -FVj ( e ) — 

(e) — e have zero mean. In other words, w,,- = A-^pi ipe 1 — M;Pz 1, for 
£ = 2, . . . , d have zero mean, where we used Py = (pe, m )- The entries pi^i and 
are even uncorrclated, for I > 1, so that 

E(P2,i) = 0, E(pi,ip 2 ,i) = 0. 

We have —1 < P2,i < 1 and its distribution is symmetric around (multiply 
the first row of Py. with —1 and the first column too. This is done by UPy^U*, 
where U is the identity but the first entry is —1). This way, we see that any 
value of pi t i corresponds to an entry j»2,i and also its negative. Thus, we have 
E^.il.Ej) = and E(pi i ip2,il.E,-) = 0, so that the entries of y~j = yl^ have 
zero mean except for the first entry. This implies with the identities ||Eyj|| = 
\®(Vj -Vi,e)\ = \E(yj,e)l E c\ and (01]) that 

IIE^II < y / P(^) v /E|(y J , e )P < V2e^e-W^E\( yj ,eW. (C.l) 

Next, we must check that E\(yj,e)\ 2 is bounded. If we apply then we 

obtain that 

niy^l^E^jPyMf-^PvMf-i) 2 

is bounded by a constant, so that (|C.1|) implies (c). □ 

Proof of Proposition \5.12l The vector Bernstein inequality as stated in (38l . The- 
orem 12] yields, for J2]=i ^Il2/j'l| 2 — ^ an( i f° r au t ^ A. I max j=i,...,n \\Vj\ that 

n 

IP ( || yj - nEy\\ >\/A + t) < cxp(-i 2 /4^)- 
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We apply our estimates in Lemma [Appendix C.l so that replacing t/n with t 



yields the following: for A = end and for alH < y f ^3, 



1 ™ l~d 
F (\\-^2vj - E Vj\\ > Jc-+t) < exp(-?ii 2 /4cd). 

i=i 

Therefore, we also derive 

-^2Vj\\ > Jc- +t + cv / 2e fe /2 e -fc/3 T ) < exp(-nt 2 /4cd). 



n ■ 



We choose /3 7 sufficiently large, i sufficiently small, and c 7 sufficiently large, 
such that, 

\ c- + t + c\/ 2e k / 2 e~ kl3 -' < — for all n > &vd, 
V n ~ 2V2 ~ 

which concludes the proof of Proposition 15.121 □ 



Appendix D. Proofs for Section 

Proof of Lemma \5.14\ By grouping P arrs °f two, the estimates (|B.1[) 

imply 

V d2P \\P ^ (fc/2)2 ^ 

_d 2 /fc(fc + 2) d 2 (fc + 4)(fc + 6) d 2 (fc + 4p-4)(fc + 4p- 2) 

" k 2 d(d + 2) ' k 2 (d + 4)(d + 6) " fc 2 (d + 4p - 4)(d + 4p - 2) 
< 2 2 f (p!) 2 . □ 
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